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C? (54) Title: TECHNIQUE FOR MANIPULATING MEDICAL IMAGES 

o 

(57) Abstract: The invention and the embodiments described in this invention provide new techniques for manipulating digital im- 
ages and is particularly useful for extracting tissues (i.e., assigning tissue boundary locations) from medical images- These techniques 
can be applied to diagnosing arthritis and for monitoring disease progression or response to therapeutic intervention. The invention 
>^ provides for means to extract the articular cartilage from medical images for analysis purposes. 
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TECHNIQUE FOR MANIPULATING MEDICAL IMAGES 

BACKGROUND OF THE INVENTION 

MR imaging is currently the most accurate technique for assessing the articular 
cartilage non-invasively in vivo. A large array of different pulse sequences can be used for 

5 imaging articular cartilage. However, at present, the diagnosis of cartilage loss is based 
mostly on qualitative, visual analysis by the radiologist One of the major obstacles in 
evaluating patients with osteoarthritis has been the lack of accurate and reproducible 
quantitative image processing and analysis techniques for monitoring progression of 
cartilage loss and response to medical or surgical therapy. 

10 Some investigators reported the use of three-dimensional reconstruction of the 

articular cartilage with subsequent volumetric quantification of the entire cartilage surface. 
In one study, cartilage was segmented from the surrounding tissues using a signal intensity 
based thresholding technique applied to magnetization transfer subtraction images or fat- 
saturated Tl-weighted images [1]. Since some of the adjacent tissues demonstrated signal 

15 intensity values overlapping those of articular cartilage, additional manual disarticulation of 
the cartilage was performed on selected image slices. Knees were imaged repeatedly. 
Cartilage volumes determined from the 3D reconstructions of MT subtraction and Tl- 
weighted fat-saturated images were correlated to values obtained with water displacement of 
surgically retrieved tissue. They reported an intra-observer reproducibility error of 0.20-0.65 

20 mL (3.6%-6.4%) for MT subtraction imaging and 0.21-0.58 mL (4.2%-6.4%) for Tl- 
weighted fat-saturated imaging [1]. Interobserver error was less than 0.62 mL and 7.8%. In a 
subsequent study involving the metacarpophalangeal joints, they found a reproducibility 
error of 5.2% (95% confidence interval 2.9% to 7.6%) for metacarpal cartilage and 9.9% 
(5.4% to 15.1%) for proximal phalangeal cartilage [2]. They concluded that three- 

25 dimensional data analysis of MR images allows reproducible volumetric quantification of 
articular cartilage in the knee and metacarpophalangeal joints. 

Piplani et al. refined this approach by combining signal intensity based thresholding 
of the articular cartilage with a connected-components or seed growing algorithm thereby 
obviating the need for manual disarticulation of the cartilage in those areas where adjacent 

30 tissues demonstrated overlapping signal intensities [3]; 
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Stammberger et al. used B-spline snakes for semi-automated segmentation of 
articular cartilage [4J. A continuous and smooth parametric curve that depends on a number 
of control points is fit around the object by means of minimizing different energy terms. 
These energy terms control the smoothness of the curve and its attraction to certain image 
5 features, e.g. high graylevel gradients, causing it to act much like a rubber band. 

Lynch et al. demonstrated a variation of the snake algorithm, in which the spline is 
adjusted to minimize costs calculated from Gauss and Canny filter responses [5]. The user 
initializes the system by selecting different control points in the medial and lateral tibio- 
femoral and the patello-femoral compartments. These control points are subsequently 
10 automatically adjusted as far as possible. K 

However, at present, there are no techniques available that perform reliably when 
used for segmentation of cartilage affected by advanced osteoarthritis. In these cases, MR 
images typically show a high degree of texture inhomogeneity of the cartilage, irregular and 
interrupted contours, and low contrast between the cartilage and surrounding tissue. These 
1 5 situations require a different technique for segmentation of the cartilage. 

We developed a system for the calculation of the 3-dimensional cartilage thickness 
that is based on a 3D Euclidian distance transformation (EDT). For a given set of feature 
points in a binary volume, the EDT computes the distance to the closest feature point for 
each non-feature point of the volume. By using the points on the cartilage-bone interface 
20 (inner cartilage surface, ICS) as feature points, the EDT measures the distance to the closest 
voxel on the ICS for all other points, including the ones on the outer cartilage surface 
(OCS), resulting in a truly three-dimensional distance value determined normal to the ICS 
(Fig. 1). 

SUMMARY OF THE INVENTION 

25 The general purpose of the invention and the embodiments described in this 

invention is to provide new techniques for extracting tissues from medical images. These 
techniques can be applied to diagnosing arthritis and for monitoring disease progression or 
response to therapeutic intervention. 

In one embodiment, the invention provides for means to estimate 

30 cartilage layers in articular joints such as the knee joint using magnetic resonance imaging 
(MRT) scans. In another embodiment, the invention provides for means to estimate the 
thickness distribution of articular cartilage layers using an MRI scan. In another 
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embodiment, the invention provides for means to measure volume, and thickness 
distribution of specific volumes of interest (VOIs) in an MRI scan. In another embodiment, 
the invention provides for means to compare baseline and follow-up MRI scans of a patient. 
In another embodiment, the invention provides for means to identify the articular cartilage 
5 in an image, such as an MRI. hi another embodiment, the invention provides for means to 
extract the articular cartilage from medical images for analysis purposes. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 demonstrates a 2D MRI (3D spoiled gradient echo. The 2D MRI 
demonstrates a Ml thickness cartilage defect in the posterior lateral femoral condyle 
10 (arrows)> as well as texture inhomogeneity 

Figure 2 demonstrates the distance d between the inner cartilage surface (ICS) and 
the outer cartilage surface (OCS), as used, for example, for the new live wire feature. 

Figure 3 shows a schematic of the use of an aspect of the invention, in this example 
used with the Live Wire algorithm. A. low contrast between OCS and surrounding tissue; 
15 bel b has low cost since gradients /? and y are almost parallel. B. High contrast between 
OCS and surrounding tissue; b has low cost due to high magnitude of gradient p . 

Figure 4 shows a schematic of bel b and its neighborhood of pixels p, q, t, u, v, w. 

PREFERRED EMBODIMENTS 

In one embodiment the invention provides for a method of manipulating an image 
20 which method comprises extracting a tissue from said image wherein said extraction is 

facilitated with use of knowledge of the thickness, curvature, shape, dimensions or contour 
of said tissue. The tissue that can be extracted in the fashion can be bone or cartilage. 

In another embodiment of the invention, knowledge of the thickness, curvature, 
shape, dimensions or contour of said tissue can be obtained from other subjects. These 
25 subjects can be age and sex matched with the patient. These subjects can be weight 

matched. These subjects can also have similar medical histories, for example a tear of the 
anterior cruciate ligament. 

In another embodiment of the invention, knowledge of the thickness, curvature, 
shape, dimensions or contour of said tissue can be obtained from a reference database. This 
30 reference database can have bee generated in other age, sex, weight, or race matched 
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individuals. Individuals can also have similar medical histories, for example a tear of the 
medial meniscus. 

_ In another embodiment of the invention, knowledge of the thickness, curvature, 
shape, dimensions or contour of said tissue can be obtained from information derived from 
the same subject. For example, in segmenting arthritic or normal cartilage in a medical 
image, said information can be cartilage thickness, cartilage curvature or cartilage shape in a 
contralateral joint. Alternatively, said information on cartilage thickness, cartilage curvature 
or cartilage shape can be obtained in the same joint. 

In another embodiment, the thickness, curvature, shape, dimensions or contour of 
another tissue, preferably a tissue adjacent to the tissue of interest, can be used to facilitate 
the segmentation or extraction of the tissue of interest For example, in segmenting arthritic 
or normal cartilage in a medical image, the shape or contour of the subchondral bone can 
provide an estimate of the expected contour of the articular cartilage. 

In another embodiment of the invention, knowledge of the thickness, curvature, 
shape, dimensions or contour of said tissue can be used to verify plausibility and correctness 
of the manipulation or segmentation of said tissue. For example, in segmenting normal 
cartilage in a medical image, a cartilage thickness or dimensions calculated from the 
segmentation that far exceed reference values from other patients matched by weight and 
height or values calculated from a baseline exam are likely to be incorrect. 

In another embodiment, 2D MRI images can be used to create a 3D map of the 
articular cartilage thickness using (for example) color coding to map thickness on a pixel- 
by-pixel basis. This can be displayed, for example, along the 3D surface of the articular 
cartilage. 

This invention can be applied to any joint in a human or mammal, for example a 
knee joint or a hip joint. This invention can be particularly useful in humans suffering from 
arthritis and cartilage loss. 

Another aspect of the invention is a method for assessing the condition of cartilage 
in a joint of a human, which method comprises 

(a) electronically transferring an electronically-generated image of a cartilage of 
the joint from a transferring device to a receiving device located distant from 

i the transferring device; 

(b) receiving the transferred image at the distant location; 

(c) extracting said cartilage from said transferred image 
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(d) converting the transferred image to a normal or degeneration pattern Qf the 
cartilage; 

(e) storing said normal or degeneration pattern of cartilage in a database, and 

(f) transmitting said normal or said degeneration pattern to a site for analysis. 

5 The information stored in the database on said normal and degeneration patterns can 

be used to facilitate said extraction of cartilage from said transferred images in other patients 
or to verify plausibility and correctness of said extraction of cartilage from said transferred 
images in other patients. 

Another aspect of the invention is a method for assessing the condition of cartilage 

10 in a joint of a human, which method comprises 

(a) extracting said cartilage from said image 

(b) electronically transferring an electronically-generated image of said extracted 
cartilage of the joint from a transferring device to a receiving device located 
distant from the transferring device; 

15 (c) receiving the transferred image at the distant location; 

(d) converting the transferred image to a normal or degeneration pattern of the 
. cartilage; 

(e) storing said normal or degeneration pattern of cartilage in a database, and 

(f) transmitting said normal or said degeneration pattern to a site for analysis. 
20 The information stored in the database on said normal and degeneration patterns can 

be used to facilitate said extraction of cartilage from said transferred images in other patients 
or to verify plausibility and correctness of said extraction of cartilage from said transferred 
images in other patients. 

The live wire algorithm that was introduced by Falcao et al. [6] can provide a 

25 flexible general framework well-suited for adaptation for the purpose of segmenting 

osteoarthritic cartilage. It can evaluate different features of the oriented edges between two 
pixels (boundary element, bel) and finally can calculate a single cost value for every bel 
based on these features. For a starting pixel P that is selected by the user with the mouse, the 
system can calculate the bel path with the least sum of costs from each image pixel to P. 

30 When the user moves the mouse cursor, the system can now draw the calculated path from 
the current mouse position. The current path can be frozen as p v art of the object contour by 
the user. Thus, the object contour is assembled from a number of live wire segments. 
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The live wire algorithm as presented by Falcao et al. is, however, too general to be 
effective in the segmentation of osteoarthritic cartilage. The current invention can be used to 
enhance and extend the live wire algorithm, thus providing for means to efficiently segment 
cartilage from MR images in patients with severe osteoarthritis. 
5 In one embodiment of the invention, a live wire technique can be used to segment 

the cartilage. Various features can be used alone or in combination to help segment the 
articular cartilage with the live wire technique. Such features include, but are not limited to: 
L Grayvalue of the pixel on the left side of thebel 
2. Grayvalue of the pixel on the right side of the bel 

10 3. Magnitude of the graylevel gradient across the bel 

4. Distance of the closest bel in the previously segmented slice 

For example, the distance of the outer cartilage surface (OCS) to the inner cartilage 
surface (ICS) can be used as an aid in segmenting the cartilage. 

Prior knowledge that femoral cartilage, for example, usually does not exceed a 

15 thickness of approximately 4 mm can be used with this feature. If the cartilage-bone 

interface (ICS) is segmented first in a slice, which can usually be easily done on sagittal MR 
images due to high contrast between bone and cartilage, this information can be helpful in 
regions of low contrast between OCS and surrounding tissue. The feature value is the 
distance of the bel to the closest bel on the ICS (see Fig. 2). For distance calculation, a 

20 Euclidean distance transform algorithm can be used. The cost can be calculated with a 

modified hyperbolic function that increases from 0 to 1 above a thickness value taken from 
a reference database. This feature can prevent the OCS contour from moving too far away 
from the ICS contour. 

In another embodiment of the invention, the direction of the graylevel gradient of the 

25 OCS compared with the gradient of the corresponding section of the ICS can be used to help 
in segmenting the cartilage. 

Except in the periphery of the cartilage where the ICS and the OCS meet and in 
regions of defective cartilage the OCS is usually approximately parallel to the ICS. This 
knowledge can be used to combine magnitude and direction of the graylevel gradient in 

3 0 order to help finding the contour in regions of low contrast between the OCS and 

surrounding tissue. If b is a boundary element under consideration for the OCS and c the 
closest bel on the previously segmented ICS, and f3 and fare the vectors of the graylevel 
gradient across b and c, the value / for this feature can be computed as follows: 
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In this equation, the first factor is the inverse of the gradient magnitude across b. The 

second factor includes the absolute value of the cosine of the angle y/ between and f 

and has a low value when the vectors are nearly parallel since |cos y/\ is close to 1 . Thus, the 

5* lower the gradient magnitude across b 9 the more weight is given to its direction. The cost 
function is a linear mapping within a predetermined interval I of feature values. Values 
outside of I are mapped to 1. 

Thus, within I the cost for b increases with increasing deviation of the direction of 

gradient p from the direction of the gradient on the opposing ICS. A high degree of 
10 parallelism between and f results in a high probability of b being included in the 

contour (see Fig. 3 A). In effect, this means that the ICS is taken as a template for the OCS 

where the OCS is not clearly recognizable. In areas of high contrast between the OCS and 

surrounding tissue, however, this is not necessary, so that the gradient direction can be 

weighted less (see Fig. 3B). This also makes the detection of cartilage defects possible. 
15 In another embodiment, different boundary element features may be considered 

together. Their relative weighting can be the same for each slice or can be changed 

depending on regional variations in one or more of these features. 

The invention provides for means to reliably segment articular cartilage from MR 

images that has been damaged due to advanced osteoarthritis. In one embodiment, the 
20 invention will be used to determine the total volume of an articular cartilage layer. 

In another embodiment, the invention will be used to calculate the 3-dimensional 

thickness distribution of an articular cartilage layer. 

In another embodiment, the invention will be used to calculate a volume of interest 

that is targeted over an area of diseased cartilage rather than including the entire surface of 
25 the articular cartilage. 

In another embodiment, the invention provides for means to compare baseline and 

follow-up studies of a patient This will be done by segmenting the articular cartilage in both 

studies using the invention, and subsequently registering the segmented cartilage objects by 

means of a surface registration technique. : 
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In another embodiment; the invention provides means to verify plausibility and 

correctness of said thickness and volume calculations. 

The techniques provided in this invention provide means to accurately and reliably 

extract cartilage from medical images. For example, the techniques described in this 
5 invention provide means to accurately and reliably segment cartilage from MR scans that 

has been destroyed or deformed due to osteoarthritis; Existing cartilage segmentation 

methods do not perform reliably and accurately in case of diseased cartilage. The invention 

improves the diagnostic capabilities of cartilage quantification methods that can be used for 

the assessment of arthritis, for example in patients undergoing treatment with a 
10 chondroprotective or a chondroregenerative agent. The invention also facilitates the 

segmentation process for the user, thus reducing the amount of time necessary to segment 

cartilage from the MR or other medical images. 

Someone skilled in the art will easily recognize other means of practicing the 

invention. For example, the specific cost mapping functions used to calculate the feature 
15 costs, and its parameters can be easily modified. Variations of formula 1, leading to similar 

results, are evident 

EXAMPLES 

The examples below are exemplary of how aspects of the invention can be practiced. 
20 The examples are in no way meant to be limiting of the invention. Someone skilled in the 
art will easily recognize other means of practicing the invention. 

In one example, a live wire algorithm is used for segmentation of osteoarthritic 
cartilage. Several features can be used alone or in combination to help segment the articular 
cartilage with the live wire technique. The features in this example include: 
25 L the grayvalue of the pixel on the left side of thebel 

2. the grayvalue of the pixel on the right side of the bel 

3. the magnitude of the graylevel gradient across the bel 

4. the distance of the closest bel in the previously segmented slice 

5. the distance of the outer cartilage surface (OGS) to the inner cartilage surface (ICS). 
30 Prior knowledge that femoral cartilage, for example, usually does not exceed a 

thickness of approximately 4 mm can be used with this feature. The feature value is the 
distance of the bel to the closest bel on the ICS (see Fig. 2). For distance calculation, a 
Euclidean distance transform algorithm can be used. The cost is calculated with a modified 
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hyperbolic function that increases from 0 to 1 above a reference thickness value. This 
feature can prevent the OCS contour from moving too far away from the ICS contour. 

Listing of preferred embodiments of the invention 

As will be apparent to those of skill in the arts to which the current invention 
5 pertains, there are numerous elements involved in the practice of the invention, and those 
elements can be combined in different maimers to provide multiple embodiments of the 
invention. A non-exhaustive list of preferred embodiments is provided to exemplify 
practice of the invention. This list is primarily directed to medical applications, particularly 
those that relate to the determination of the outer cartlidge surface of an articular joint, as 
10 detennination of such a surface from a digital image is a particularly difficult challenge for 
which the present invention is well suited. 

1 . A method of manipulating an image which method comprises 

a. detennining the gray value of a pixel on the left side of a boundary element 

b. determining the gray value of a pixel on the right side of a boundary element 

15 c. determining the magnitude of the graylevel gradient across the boundary element 

d. detennining the distance of the closest boundary element in the previously 
manipulated image slice, 

whereby determining the distance of the closest boundary element in the previously 
manipulated image slice improves the accuracy of the manipulation. 

20 2. The method of embodiment 1 wherein the image is an MRI and the method is used 
to locate the bone cartilage interface. 

3. The method of embodiment 1 wherein the image is an MRI and the method is used 
to locate the external cartilage surface. 

4. The method of embodiments 2 and 3 wherein the cartilage is normal. 

25 5. The method of embodiments 2 and 3 wherein the cartilage is diseased. 

6. A method of manipulating an image which method comprises extracting a tissue 
from the image wherein the extraction is facilitated with use of knowledge of the thickness, 
curvature, shape or contour of the tissue. 
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7. The method of embodiment 6 wherein the tissue is bone. 

8. The method of embodiment 6 wherein the tissue is cartilage. 

9. The method of embodiment 6 wherein the knowledge of the thickness, the curvature, 
or the contour is obtained from information derived from other subjects. 

5 10. The method of embodiment 9 wherein the other subj ects are age and sex matched 
with the patient. 

11. The method of embodiment 9 wherein the other subjects are weight matched. 

12. The method of embodiment 6 wherein the knowledge is obtained from information 
derived from the same subject 

1 0 13. The method of embodiment 6 wherein the information is cartilage thickness, 
cartilage curvature or cartilage shape in a contralateral joint. 

14. The method of embodiment 6 wherein the information is cartilage thickness, 
cartilage curvature or cartilage shape in the same joint. 

15. The method of embodiment 6 wherein the knowledge is curvature or shape of the 
15 bone in the same joint. 

1 6. The method of embodiment 1 5 wherein the bone is subchondral bone. 

17. The method of embodiment 16 wherein the curvature or the shape of the 
subchondral bone is used to estimate the curvature or shape of the overlying articular 
cartilage. 

20 18. The method of embodiment 6 wherein the image is an image of the knee joint. 

19. The method of embodiment 1 8 wherein the joint has arthritis. 

20. The method of embodiment 19 wherein the arthritis has caused cartilage loss. 

21. A method of manipulating an image which method comprises segmenting the image 
wherein the segmentation is facilitated with use of knowledge of an expected range of the 

25 thickness, curvature, shape, dimensions or contour of at least one tissue. 

22. The method of embodiment 2 1 wherein the tissue is bone. 

23. The method of embodiment 21 wherein the tissue is cartilage. 
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24. The method of embodiment 2 1 wherein the knowledge of the thickness, the 
curvature, the shape, the dimensions or the contour is obtained from information derived 
from other subjects. 

25. The method of embodiment 24 wherein the other subjects are age and sex matched 
5 with the patient 

26. The method of embodiment 24 wherein the other subjects are weight matched. 

27. The method of embodiment 2 1 wherein the knowledge is obtained from information 
derived from the same subject 

28. The method of embodiment 21 wherein the information is cartilage thickness, 
10 cartilage .curvature, cartilage contour or cartilage shape in a contralateral joint. 

29. The method of embodiment 21 wherein tiie information is cartilage thickness, 
cartilage curvature, cartilage contour or cartilage shape in the same joint. 

30. The method of embodiment 21 wherein the knowledge is curvature, contour or shape 
of at least one bone in the same joint. 

15 31. The method of embodiment 30 wherein the bone is subchondral bone. 

32. The method of embodiment 31 wherein the curvature, the contour or the shape of the 
subchondral bone is used to estimate the curvature, contour or shape of the overlying 
articular cartilage. 

33. The method of embodiment 21 wherein the image is an image of the knee joint 
20 34. The method of embodiment 33 wherein the joint has arthritis. 

35. The method of embodiment 34 wherein the arthritis has caused cartilage loss. 

36. The method of embodiment 32 wherein the curvature, the contour or the shape of the 
subchondral bone is used to estimate the curvature, contour or shape of the overlying 
articular cartilage at the interface of the subchondral bone with the overlying articular 

25 cartilage. 

37. The method of embodiment 32 wherein the curvature, the contour or the shape of the 
subchondral bone is used to estimate the curvature, contour or shape of the overlying 
articular cartilage at the external, articular surface. 

38. A method of manipulating a medical image which method comprises 
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a. determining the gray value of a pixel on the left side of a boundary element 

b. determining the gray value of a pixel on the right side of a boundary element 

c. determining the distance of a boundary element to the inner cartilage surface 
(ICS) 

5 whereby the inner cartilage surface is located in the joint of a human and whereby 
determining the distance of the boundary element to the inner cartilage surface (ICS) 
improves the accuracy of the manipulation. 

39. The method of embodiment 38 wherein the image is an MRI and the method is used 
to locate the outer cartilage surface. 

10 40. The method of embodiment 39 wherein the cartilage is normal. 

41 . The method of embodiment 39 wherein the cartilage is diseased. 

42. The method of embodiment 38 wherein the manipulation includes a live wire 
algorithm. 

43. The method of embodiment 42 wherein the live wire algorithm uses a cost function 
15 that employs a modified hyperbolic function that increases from 0 to 1 when the distance 

from the inner cartilage surface to the outer cartilage surface increases above a reference 
value in the femoral condyles. 

44. The method of embodiment 42 wherein the live wire algorithm uses a cost function 
that employs a modified hyperbolic function that increases from 0 to 1 when the distance 

20 from the inner cartilage surface to the outer cartilage surface increases above a reference 
value in the tibial plateau. 

45. The method of embodiment 42 wherein the live wire algorithm uses a cost function 
that employs a modified hyperbolic function that increases from 0 to 1 when the distance 
from the inner cartilage surface to the outer cartilage surface increases above a reference 

25 value in the patella. 

46. The method of embodiment 38 wherein the manipulation includes a snake algorithm. 

47. A method of manipulating an image which method comprises 

a. determining the gray value of a pixel on the left side of a boundary element 

b. detennining the gray value of a pixel on the right side of a boundary element 
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c. detennining the distance of the closest boundary element in the previously 
manipulated image slice, 

whereby determining the distance of the closest boundary element in the previously 
manipulated image slice improves the accuracy of the manipulation. 

5 48. The method of embodiment 47 wherein the image is an MRI and the method is used 
to locate the bone cartilage interface. 

49. The method of embodiment 47 wherein the image is an MRI and the method is used 
to locate the external cartilage surface. 

50. The method of embodiments 48 and 49 wherein the cartilage is normal. 
10 51. The method of embodiments 48 and 49 wherein the cartilage is diseased. 

52. A method for assessing the condition of cartilage in a joint of a human, which 
method comprises 

(a) electronically transferring an electronically-generated image of a cartilage of 
the joint from a transferring device to a receiving device located distant from 
15 the transferring device; 

. (b) receiving the transferred image at the distant location; 

(c) extracting the cartilage from the transferred image 

(d) converting the transferred image to a normal or degeneration pattern of the 
cartilage; and 

20 (e) transmitting the normal or the degeneration pattern to a site for analysis. 

53. The method of embodiment 52 wherein the normal or the degeneration pattern is 
stored in a database. 

54. The method of embodiment 53 wherein the database can be used to facilitate the 
extraction of the cartilage from the transferred images in other patients. 

25 55. The method of embodiment 53 wherein the database contains information on 
subjects' age, sex, race, weight or medical condition. 

56. The method of embodiment 52 wherein after step (e), the degeneration pattern is 
displayed as a three-dimensional image. 
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57. The method of embodiment 56 wherein the joint is a knee joint 

58. The method of embodiment 52, 56 or 57 wherein all steps are carried out at an initial 
time (Tl) and are carried out again at a later time (12). 

59. The method of embodiment 58 wherein the assessment includes an analysis of the 
5 degree of degeneration of the cartilage between Tl and T2. 

60. The method of embodiment 52 wherein the electronically-generated image of a 
cartilage is obtained by a magnetic resonance imaging (MKT) technique. 

61. A method of embodiment 60, wherein the MRI technique provides a biochemical 
description of the cartilage. 

10 62. A method of embodiment 60, wherein the MRI technique provides a volumetric 
description of the cartilage. 

63. The method of embodiment 60 wherein the MRI technique results in a three- 
dimensional image of the cartilage. 

64. The method of embodiment 63 wherein the MRI technique first obtains a series of 
15 two-dimensional views of the joint, which are then mathematically integrated to give a 

three-dimensional image. 

65. The method of embodiment 60 wherein the MRI technique employs a gradient echo, 
spin echo, fast-spin echo; driven equilibrium fourier transform, or spoiled gradient echo 
technique. 

20 66. The method of embodiment 60, wherein the assessment of the condition of cartilage 
in a joint of a human is followed by one or more assessments of the condition of the same 
cartilage at a later point in time. 

67. A method for assessing the condition of cartilage in a joint of a human, which 
method comprises 

25 (a) extracting the cartilage from an electronically-generated medical image 

(b) electronically transferring the electronically-generated image of a cartilage of 
the joint from a transferring device to a receiving device located distant from 
the transferring device; 

(c) receiving the transferred image at the distant location; 

14 
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(d) converting the transferred image to a normal or degeneration pattern of the 
cartilage; and 

(e) transmitting the normal or the degeneration pattern to a site for analysis. 

68. The method of embodiment 67 wherein the normal or the degeneration pattern is 
5 stored in a database. 

69. The method of embodiment 68 wherein the database can be used to facilitate the 
extraction of the cartilage from the transferred images in other patients. 

70. The method of embodiment 68 wherein the database contains information on 
subjects* age, sex, race, weight or medical condition. 

10 71. The method of embodiment 67 wherein after step (e), the degeneration pattern is 
displayed as a three-dimensional image. 

72. The method of embodiment 67 wherein the joint is a knee joint 

73. The method of embodiment 67, 71 or 72 wherein all steps are carried out at an initial 
time (Tl) and are carried out again at a later time (T2). 

15 74. The method of embodiment 73 wherein the assessment includes an analysis of the 
degree of degeneration of the cartilage between Tl and T2. 

75. The method of embodiment 67 wherein the electronically-generated image of a 
cartilage is obtained by a magnetic resonance imaging (MRI) technique. 

76. A method of embodiment 75, wherein the MRI technique provides a biochemical 

20 description of the cartilage. 

j 

77. A method of embodiment 75, wherein the MRI technique provides a volumetric 
description of the cartilage. 

78. The method of embodiment 75 wherein the MRI technique results in a three- 
dimensional image of the cartilage. 

25 79. The mfethod of embodiment 78 wherein the MRI technique first obtains a series of 
two-dimensional views of the joint, which are then mathematically integrated to give a 
three-dimensional image. 
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80, The method of embodiment 75 wherein the MRI technique employs a gradient echo, 
spin echo, fast-spin echo, driven equilibrium fourier transform, or spoiled gradient echo 
technique. 

81. The method of embodiment 75, wherein the assessment of the condition of cartilage 
5 in a joint of a human is followed by one or more assessments of the condition of the same 

■ v cartilage at a later point in time. 

Detailed Specific Example 

10 METHODS 

Image Segmentation 

The live wire algorithm finds the contours of an ^object by evaluation of a set of features for 
each boundary element A boundary element (bet) is the oriented edge between two pixels 
(see Figure 4). Using an individual cost function for each feature, the feature values are 

1 5 converted into cost values between 0 and 1 . Finally, the weighted sum of all feature costs 
results in a single joint cost value between 0 and 1 for each bel b that expresses the 
likelihood of b being part of the cartilage boundary [6] . This determines whether the pixels 
on either side of the bel are assigned to one tissue or the other that form the boundary. To 
determine the contour of a cartilage object, the user chooses a starting pixel P with a mouse 

20 click. Subsequently, the system calculates the bel path with the least sum of costs from each 
image pixel to P using a dynamic programming scheme. When the user moves the mouse 
cursor, the system can now display the calculated path from the current mouse position to P 
in real time. The current path can be frozen as part of the cartilage contour by the user. 
Thus, the cartilage contour in each slice is assembled from a number of frozen live wire 

25 segments. 

The live wire contour should be drawn around the cartilage in a consistent direction. In 
the following it is assumed that the direction has been chosen by the user such that the 
cartilage is always to the right of a bel. We are currently using the following boundary 
element features and cost functions: 

i 

30 1 . Grayvalue of pixel p: The grayvalue of pixel p (see Figure 4), which is supposed to 
lie outside the cartilage, is mapped to a cost value using an inverted Gaussian 
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^ function with the mean and standard deviation as parameters that have to be chosen 
appropriately. Grayvalues close to the mean are converted to low cost values. 

2. Grayvalue of pixel q: The same process is applied to pixel q, which is assumed to be 
a cartilage pixel. These features attract the contour to pixels with grayvalues close 
the chosen means. 

3. Magnitude of the graylevel gradient across b is computed according to equation 2. 
The cost function is a thresholding function that is 0 inside and 1 outside a given 
interval. This prevents the contour from locking onto the cartilage-bone interface of 
the tibia when segmenting the femoral outer cartilage surface. 



10 



4. Distance ofb to closest bel in previously segmented slice: The contour is drawn 
towards a previously segmented contour in an adjacent slice. Thus, one carefully 
segmented slice can provide useful information for subsequent slices. 

The distance is calculated by means of a fast Euclidean distance transform (EDT) 
15 algorithm [12]. The cost increases linearly with the distance. 

5. Distance of outer cartilage surface (OCS) to inner cartilage surface (ICS): Prior 
knowledge that femoral cartilage, for example, typically does not exceed a thickness 
of approximately 4 mm [7] is incorporated as another feature. If the cartilage-bone 
interface (ICS) is segmented first in a slice, which can usually be easily done in 

20 sagittal MR images due to high contrast between bone and cartilage, this information 

is helpful in regions of low contrast between OCS and surrounding tissue. 
For distance calculation the same EDT algorithm as in feature 4 is used. The cost is 
calculated with a modified hyperbolic function that increases from 0 to 1 above 
4mm. This feature prevents the OCS contour from moving too far away from the 

25 ICS contour. 

6. Direction of graylevel gradient of OCS compared with the gradient of the 
corresponding section of ICS: Except in the periphery of the cartilage where the ICS 
and the OCS meet and in regions of defective cartilage the OCS is usually 
approximately parallel to the ICS. This knowledge is used to combine magnitude 

30 and direction of the graylevel gradient in order to help finding the contour in regions 
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of low contrast between the OCS and surrounding tissue. The lower the gradient 
magnitude across a bel b on the OCS, the more weight is given to its direction. The 
cost for b increases with increasing deviation of the direction of its gradient from 

the direction of the gradient ^ on the opposing ICS. A high degree of parallelism 

. 5 between ~f and ^ results in a high probability of b being included in the contour. 

In effect, this means that the ICS is taken as a template for the OCS where the OCS 
is not clearly recognizable, In areas of high contrast between the OCS and 
surrounding tissue, however, this is not necessary, so that the gradient direction can 
be weighted less. This also makes the detection of cartilage defects possible. 
10 To facilitate the selection of the Gaussian cost function parameters for the user, the 

images may be first processed with a Kuwahara smoothing filter with a 3x3 window 
size [13]. This non-linear filter reduces nbise while preserving eges in the image. 

Interobserver Reproducibility 

15. In preliminary experiments interobserver reproducibility of the proposed method was 

assessed using sagittal MR images of the knee acquired from five, patients with different 
stages of osteoarthritis (1 .5T GE Signa, fat-saturated 3D SPGR, TE 5ms, TR 60ms, flip 
angle 40°, voxel size 1.5x0.47x0.47mm 3 ). Three observers segmented the femoral cartilage 
with the live wire method, using features 1-3. Parameters measured from segmented data 

20 were total cartilage volume, maximum thickness, average thickness, and number of voxels 
on the OCS. Thickness values were calculated using a 3-dimensional EDT [8, 12]. 
Interreader reproducibility was determined as the coefficient of variation (CV%) between 
the three, observers for each patient 

Results 

25 An example of the processing steps for the cartilage segmentation is given above. 

♦ The contours of normal cartilage and areas of thinning and full thickness loss are correctly 
identified with the live wire technique 

The results of the interobserver reproducibility assessment are presented in Table 1 . 

Table 1 

30 Interobserver reproducibility (CV%). The average interobserver reproducibility is given as 
root mean square (RMS) of the CV%. 
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Patient 


Volume 


Max: thickness 


Avg. thickness 


Surface voxels 


A 


4.7% 


0.2% 


3.8% 


1.5% 


B 


13.5% 


7.6% 


10.5% 


2.7% 


c 


8 6% 


3.3% 


6 7% 


0 7% 


D 


7.7% 


4.9% 


5.7% 


3.1% 


E 


5.9% 


0.7% 


5.8% 


1.0% 


RMS 


8.6% 


4.3% 


6.9% 


2.0% 



Discussion 

The experiments on the interobserver reproducibility of segmentation of osteoarthritic 
. femoral cartilage using the proposed live wire technique yielded approximately the same 
5 results as published for similar parameters obtained from healthy volunteers with the B- 
spline snake method [11]. However, as Table 1 also shows, the variation of values obtained 
for the total cartilage volume remains high; This is partially caused by difficulties to 
identify the: correct cartilage boundaries in the peripheral regions of the femoral condyles 
where the sagittal imaging plane is nearly tangential to the cartilage surface, thus blurring 
10 the edges due to high partial volume effects. 

In general, the live wire segmentation technique proved to provide the necessary 
flexibility for segmentation of osteoarthritic cartilage. New features can easily be 
implemented and included into the system. Subjectively, the technique greatly facilitates 
segmentation compared to manual delineation of the contours. Its principal advantage lies 
15 in the evaluation of local image features in a global context, which is important for the 
correct treatment of cartilage irregularities in segmentation. 

Further improvements can be made by including methods for automatic 
determination of the cost function parameters from the images, which will additionally 
increase reproducibility. Thus, the present technique holds the potential to improve and 
20 facilitate semi-automatic segmentation of both nonnal and diseased cartilage. 
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The present invention is 'related to and can be used in combination with other 
inventions arising out of investigations in the laboratories of the present inventors, as well 
10 as in combination with many other independently developed or publically available 

techniques for the investigation and manipulation of digital images, particularly such images 
as they relate to medical images and the segmentation of tissues. Examples of applications 
related to the present invention that were developed in the laboratories of the present 
inventors can be found in the disclosures of the following patent documents: 

PCT Publication No. WO 00/35346, published 22 June, 2000, 
entitled "Assessing the Condition of a Joint and Preventing Damage/' 

U.S. application serial No: 60/232,639, filed 14 September 
2000, entitled "New Techniques for Manipulating Medical Images." 

U.S. application serial No. 60/232,637, entitled "Assessing 
the Condition of a Joint and Assessing Cartilage Loss." 

U.S. application serial No. 09/662,224, entitled "Assessing 
the Condition of a Joint and Devising Treatment" 

All publications and patent applications mentioned in this specification are herein 
incorporated by reference to the same extent as if each individual publication or patent 
25 application was specifically and individually indicated to be incorporated by reference. 

The invention now being fully described, it will be apparent to one of ordinary skill 
in the art that many changes and modifications cin be made thereto without departing from 
the spirit or scope of the appended claims. . 
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CLAIMS 

We claim: 

1 . A method of assessing a digital image to detennine a boundary between first and second image 
components, which comprises: 

(a) selecting a first boundary element between two pixels being evaluated for assignment 
to the first or second image component; 

(b) determining a first value of a first pixel on one side of the first boundary element; 

(c) detennining a second value of a second pixel on the other side of the first boundary 
element; " 

(d) comparing the first and second value to determine magnitude of a gradient across the 
first boundary element and assigning a gradient boundary value to the first boundary element; 

(e) determining relative locations of (1) the first boundary element and (2) a reference 
element selected from the group consisting of (2a) a reference boundary element previously 
assigned as separating a pixel assigned to the first image component from a pixel assigned to the 
second image component and (2b) a reference second-boundary element previously assigned as 
separating a pixel assigned to one of the first and second image components from a third image 
component, the second-boundary element having a known, statistically significant distance or 
angle relationship relative to an actual boundary between real components represented by the first 
and second image components, and assigning a location boundary value to the first boundary 
element based on distance or angle of the relative locations from each other; 

(f) mathematically combining the gradient boundary value and the location boundary 
value to provide a first potential boundary value for the first boundary element; 
repeating steps (a)-(f) for a second boundary element to obtain a second potential boundary 
value; and 

(g) assigning the first boundary element or the second boundary element as dividing the 
first image component from the second image component, depending on relative potential 
boundary values of the first and second boundary elements. 
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2. The method of claim 1, wherein said image is a magnetic resonance image, an ultrasound 
image, or an X-ray image and said method is used to locate a bone/cartilage interface or an 
external cartilage surface. 

3. The method of claim 2, wherein said cartilage is diseased or damaged. 

4. The method of claim 1, wherein said method assigns said boundary element to said first or 
second image component using existing knowledge of probable shape of said first or second 
image component 

5 . The method of claim 4, wherein said digital image is of a living subject and said first and 
second image components comprise first and second tissues of said living subject. 

6. The method of claim 5, wherein said knowledge is obtained from (a) information derived from 
same-type tissue images of other subjects or (b) information derived from a different image 
obtained from said living subject or from a different region of said tissue image. 

7. The method of claim 6, wherein said other subjects are age and sex matched or weight 
matched with said living subject. 

8. The method of claim 6, wherein said information is (a) cartilage thickness, cartilage curvature 
or cartilage shape in a contralateral joint or in a different region of a first joint whose image is 
being assessed or (b) curvature or shape of a bone in said first joint 

9. Hie method of claim 8, wherein said bone is subchondral bone. 

10. The method of claim 9, wherein curvature or shape of the subchondral bone is used to 
estimate curvature or shape of overlying articular cartilage. 
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11. The method of claim 1, wherein said method is used to assess condition of cartilage in a joint 
of a human, which method further comprises: 

(a) electronically transfeiring^an electronically generated image of a cartilage of the 
joint from a transferring device to a receiving device located distant from the transferring 
device; 

(b) receiving the transferred image at the distant location; 

(c) extracting said cartilage from said transferred image using the method of claim 1 ; 

(d) converting the transferred image to a normal or degeneration pattern of the 
cartilage; and 

(e) transmitting said normal or said degeneration pattern to a site for analysis. 

12. The method of claim 11, wherein said normal or said degeneration pattern is stored in a 
database. 

13. The method of claim 1, wherein all steps are carried out at an initial time Tl and are carried 
out again at a later time T2. 

14. The method of claim 13, wherein the assessment includes an analysis of the degree of 
degeneration of the cartilage between Tl and T2. 

15. The method of Claim 1, wherein said digital image of a cartilage is obtained by a magnetic 
resonance imaging (MRI) technique. 



24 



WO 02/23483 PCT/US01/42155 



Figure! 




WO 02/23483 



2/4 



PCT/US01/42155 



Iig.2 



^*^**^»^ Bone 




r* ■ ics 

o* 

— ocs 



/ - WO 02/23483 PCT/USO 1/42 155 

» * 

4/4 



V 


p 

b 

w 


t 


w 


• r 

q 


u 



Figure 4 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property Organization 
International Bureau 

(43) International Publication Date 
21 March 2002 (2L03.2002) 




(10) International Publication Number 

PCT WO 02/023483 A3 



(51) International Patent Classification 7 : 



(21) International Application Number: PCT/US01/42155 

(22) International Filing Date: 

14 September 2001 (14.09.2001) 



(25) Filing Language: 

(26) Publication Language: 



G06T 5/00 (81) Designated States (national): AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CO, CR, CU, 
CZ, DE, DK, DM, DZ, EC, EE, ES, FI, GB, GD, GE, GH, 
GM, HR, HU, ED, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, 
LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, 
MX, MZ, NO, NZ, PH, PL, PT, RO, RU, SD, SE, SG, SI, 
SK, SL, TJ, TM, TR, TT, TZ, UA, UG, UZ, VN, YU, ZA, 
ZW. 



English 
English 



(30) Priority Data: 

60/232,637 
60/232,639 



14 September 2000 (14.09.2000) US 
14 September 2000 (14.09.2000) US 



(71) Applicant: LELAND STANFORD JUNIOR UNIVER- 
SITY [US/US] ; Suite 350, 900 Welch Road, Palo Alto, CA 
94304 (US). 

(72) Inventors: STEINES, Daniel; 3619 Park Boulevard, Palo 
Alto, CA 94306 (US). LANG, Philipp; 36 Fairlawn Lane, 
Lexington, MA 02420 (US). 



(84) Designated States (regional): ARIPO patent (GH, GM,. 
KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 
patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE,: 
IT, LU, MC, NL, PT,' SE, TR), OAPI patent (BP, BJ, CF, 
CG, CI, CM, GA, GN, GQ, GW, ML, MR, NE, SN, TD, 
TG). 

Published: 

— with international search report 

(88) Date of publication of the international search report: 

27 March 2003 



(74) Agents: NEELEY, Richard, L.; Cooley Godward LLP, 
Five Palo Alto Square, 3000 El Camino Real, Palo Alto, 
CA 94306-2155 etal. (US). 



For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette, 



(54) Title: TECHNIQUE FOR MANIPULATING MEDICAL IMAGES 




(57) Abstract: The invention and the embodiments described in this invention 
provide new techniques for manipulating digital images and is particularly useful 
for extracting tissues (i.e., assigning tissue boundary locations) from medical 
images. These techniques can be applied to diagnosing arthritis and for monitoring 
disease progression or response to therapeutic intervention. The invention provides 
for means to extract the articular cartilage from medical images for analysis 
purposes. 



INTERNATIONAL SEARCH REPORT 



lr atlonal Application No 

pTVUS 01/42155 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 7 G06T5/00 






According to International Patent Classification (IPC) or lo both national classification and IPC 




B. FIELDS SEARCHED 


Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 G06T 


Documentation searched other than minimum documentation to the extent thai such documents are included In the fields searched 


Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 

EPO-Internal 


C. DOCUMENTS CONSIDERED TO BE RELEVANT 


Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


A 


STAMMBERGER T ET AL: "Determination of 3D 
cartilage thickness data from MR imaging: 
computational method and reproducibility 
in the living" 

MAGNETIC RESONANCE IN MEDICINE, ACADEMIC 

PRESS, DULUTH, MN, US, 

vol. 41, no. 3, March 1999 (1999-03), 

pages 529-536, XP002161461 

ISSN: 0740-3194 

cited in the application 

cibstrsct 

page 530, paragraphs MATERIALS, AND .METHODS 


1-15 






i~- 




j jjjj Further documents are listed in the continuation of box C. 


| j Patent family members are listed 


In annex. 


• Special categories of cited documents : 

•A' document defining the general state of the art which is not 
considered to be of particular relevance 

*E" earlier document but published on or after the international 
filing date 

"L" document which may throw doubts on priority claim (s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

'O* document referring to an oral disclosure, use, exhibition or 
other means 

"P* document published prior to the international filing dale but 
bterthan the priority dale claimed 


T later document published after the international fiBng date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
Invention 

•X" document of particular reievance; the claimed fnverrtJon 
cannot be considered novel or cannot be considered to 
Involve an inventive step when the document is taken atone 

'Y* document of particular relevance; the claimed invention 
cannot be considered to involve an inventive step when the 
document Is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
In the art 

•&" document member of the same patent f amily 


Date of the actual completion of the international search 


Date of mailing of the international s« 


arch report 


11 October 2002 


29/10/2002 




Name and mailing address of the ISA 

European Patent Office, P.B. 581 8 Patent ban 2 
Nt-2280HVR5swijk 
Tel (+31-70) 340-2040, Tx. 31 651 epo nl. 
Fax (+31-70) 340-3016 


Authorized officer 

Gonzalez Ordonez, 0 



F«m PCTY5SA/210 (second sheet) *Juty 1992) 





INTERNATIONAL SEARCH REPORT 


I rational Application No 






pTt/US 01/42155 



C.(Continuatlon) DOCUMENTS CONSIDERED TO BE RELEVANT 



Category • I atalton of document, with indicaUon,wtiere appropriale, ol Oie relevant passages Relevant to claim No. 



A ZOHARA A. COHEN ET AL.: "Knee cartilage 1-15 

topography, thickness, and contact areas 
from MRI: in-vitro calibration and in-vivo 
measurements" 

OSTEOARTHRITIS AND CARTILAGE, 'Online! 

vol. 7, 1999, pages 95-109, XP002216548 

Retrieved from the Internet: 

<URL: http://citeseer.nj. nec.com/477079. htm 

1> 'retrieved on 2002-10-11! 

abstract 

page 97, left-hand column, line 8 
-right-hand column, line 21 

A ANDRIACCHI THOMAS P ET AL: "Methods for 1-15 

evaluating the progression of 
osteoarthritis" 

JOURNAL OF REHABILITATION RESEARCH AND:, 
DEVELOPMENT, THE SERVICE, WASHINGTON, DC, 
US, 

vol. 37, no. 2, March 2000 (2000-03), 
pages 163-170, XP002188111 
ISSN: 0748-7711 

page 167, left-hand column, line 45 - line' 
55 

A H0NGYI LI ET AL: "A boundary optimisation 1-15 

algorithm for delineating brain objects 
from CT-scans" 

NUCLEAR SCIENCE SYMPOSIUM AND MEDICAL 
IMAGING CONFERENCE, 1993., 1993 IEEE 
CONFERENCE RECORD. SAN FRANCISCO, CA> USA 
3T OCT. -6 NOV. 1993, NEW YORK, NY,: 
USA, IEEE, 

31 October 1993 (1993-10-31), pages 
' 1553-1557, XP010119366 
ISBN: 0-7803-1487-5 
page 1553, paragraph II 
page 1554, paragraph III 



torn PCT/ISA/210 (calBMKto> ol socond sheol) (July 1092) 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 
E^FADED TEXT OR DRAWING 
^BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCED) OR EXHH$rT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: ' ': . ' ■ "■ ■ ; V< ' - '^ ],';'"■:'}/.. ; " ':-y-i--V3/-;-.:; r / : ." : •' - : ' 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



